Comparison of shadow detection in a difficult scene

This material consists of 3 parts: 1) Comparison of shadow detection on a challenging case; 2) Applications of our work; 3) Extended visual comparisons of our shadow removal results. We plan to officially release our data set in 2014. To allow for repeatable evaluation for future comparison, our P-code, the user input data, output data and the scripts for ground truth evaluation and quantitative evaluations of shadow removal will also be available along with the release of our high quality data set. 1 Comparison of shadow detection in a difficult scene (a) Input (b) [3] (c) [2] (d) [6] (e) [1] (f) Ours Figure 1: Shadow detection on a challenging case using several alternative methods. The red and blue scribbles indicate shadow pixels and lit pixels respectively. Method Input Execution time Guo et. al[3] None 17.05s (MEX) Gong et. al[2] Shadow scribble 5.40s (MEX) Vezhnevets et. al[6] Both scribbles 0.04s (MEX) Chen et. al[1] Both scribbles 8.06s (script) Ours Both scribbles 0.71s (script) Table 1: Input and execution time (on MATLAB and the same machine) for shadow detection. In Fig. 1, we show sample visual and speed comparisons along with mask detection results given a challenging case compared with two state of the art shadow removal methods [2, 3], a state of the art general matting algorithm [1], and Grow-Cut segmentation [6]. The user input type and execution times are listed in Table 1. As the chessboard pattern is very shadow-like while the real shadow is light and soft, only [1] and ours can appropriately c © 2012. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms. 2 GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH detect the shadow. However, [1] is relatively less efficient by comparison and results in a noisier shadow mask. 2 Applications of our work We demonstrate the applications of our algorithm for interactive shadow editing in variable scenes. We recommend our readers to see the attached video for the demonstrations of them. Due to the fact that our method is fast enough and requires no prior training and no specific assumptions of illumination change model, our method is exclusively suitable for real-time interactive shadow editing. This offers free controls for shape, darkness and smoothness of either new or original shadows, as opposed to simple distortions and replications of original shadows in previous work [4, 5, 7]. We show six examples of interactive shadow editing based on original images in Table. 2. Original Shape Sharpness Intensity Removal Table 2: For each example, note that initial penumbra and umbra strokes have already been added. What is shown are editing operations that may be performed after this initial stage. GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH 3 3 Extended visual comparisons of our shadow removal results Due to the limitation of file size, we are only able to show some of the test cases in variable scenes and removal quality. Thus we decided to omit the results of [8] as its quantitative scores are relatively much lower and thus less interesting to compare than the others. The results of the other competitive methods in this document are in a higher resolution, but are lossy compressed. 4 GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH Original Guo [3] Gong [2] Ours GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH 5 Original Guo [3] Gong [2] Ours 6 GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH Original Guo [3] Gong [2] Ours GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTH 7 OriginalGuo [3]Gong [2]Ours 8GONG, COSKER: INTERACTIVE SHADOW REMOVAL AND GROUND TRUTHReferences[1] Qifeng Chen, Dingzeyu Li, and C Tang. Knn matting. PAMI, 35(9):2175–2188, 2013.[2] H. Gong, D. Cosker, C. Li, and M. Brown. User-aided single image shadow removal. InICME, pages 1–6, 2013.[3] R. Guo, Q. Dai, and D. Hoiem. Paired regions for shadow detection and removal. PAMI,PP(99):1–1, 2012.[4] Y. Shor and D. Lischinski. The shadowmeets the mask: Pyramid-based shadow removal.CGF, 27(2):577–586, 2008.[5] Y.-F. Su and H. H. Chen. A three-stage approach to shadow field estimation from partialboundary information. IEEE Trans. on Image Proc., 19(10):2749–2760, 2010.[6] Vladimir Vezhnevets and Vadim Konouchine. Growcut: Interactive multi-label nd imagesegmentation by cellular automata. In proc. of Graphicon, pages 150–156, 2005.[7] Tai-Pang Wu, Chi-Keung Tang, Michael S Brown, and Heung-Yeung Shum. Naturalshadow matting. ACM TOG, 26(2):8, 2007.[8] Q. Yang, K.-H. Tan, and N. Ahuja. Shadow removal using bilateral filtering. IEEETrans. on Image Proc., 21(10):4361–4368, 2012.